Mark Indicators
Mark Indicators¶
# ignore datajoint+jupyter async warnings
import warnings
warnings.simplefilter("ignore", category=DeprecationWarning)
warnings.simplefilter("ignore", category=ResourceWarning)
warnings.simplefilter("ignore", category=UserWarning)
nwb_copy_file_name = "J1620210531_.nwb"
In order to use clusterless decoding, you need to populate the UnitMarksIndicator table. This involves populating a series of tables (SpikeSorting -> CuratedSpikeSorting -> UnitMarks -> UnitMarkIndicators)
This may seem confusing at first, as the point of clusterless decoding is to avoid spike sorting. However, in order to keep the pipeline for spike sorting and clusterless mark extraction similar as possible, we use some shared steps. However, the "spike sorting" step involves just a simple thresholding operation (sorter: clusterless_thresholder) and the curation step is essentially skipped (although the entry still needs to be populated). It also make sense to only extract the mark indicator functions time periods where there is position data.
In order to simplify some of these steps, there is a function called populate_mark_indicators. In order to use it, you must first populate SpikeSortingSelection and IntervalPositionInfo.
Note that SpikeSortingSelection itself depends on:
SpikeSortingRecordingSpikeSorterParametersArtifactRemovedIntervalList.
and SpikeSortingRecording itself depends on:
SortGroupSortIntervalSpikeSortingPreprocessingParametersLabTeam
You will have to define/populate all these things before getting to the "spike sorting" step. In order to read up on these steps, it is best to refer to tutorial notebook 1_spikesorting.ipynb for the spikesorting related steps and 4_position_info.ipynb for populating IntervalPositionInfo.
For the purposes of this notebook, let's assume you've done these steps and now simply need to populate UnitMarkIndicators via SpikeSorting. Let's import the function and look at its docstring, which will tell us what inputs the function expects:
from spyglass.decoding.clusterless import populate_mark_indicators
?populate_mark_indicators
/home/edeno/miniconda3/envs/spyglass/lib/python3.8/site-packages/seaborn/rcmod.py:82: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. if LooseVersion(mpl.__version__) >= "3.0": /home/edeno/miniconda3/envs/spyglass/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. other = LooseVersion(other)
Connecting edeno@lmf-db.cin.ucsf.edu:3306
Signature: populate_mark_indicators( spikesorting_selection_keys: list, mark_param_name='default', position_info_param_name='default_decoding', ) Docstring: <no docstring> File: /stelmo/edeno/nwb_datajoint/src/spyglass/decoding/clusterless.py Type: function
From the docstring, we see that the main thing the function expects is spikesorting_selection_keys. This is a list of dictionaries, where each dictionary specifies the primary keys that SpikeSorting requires to run. It is a list of dictionaries because we typically want to extract the marks for more than one electrode at a time. Here are the primary keys required by SpikeSorting:
from spyglass.spikesorting import SpikeSorting
SpikeSorting.primary_key
['nwb_file_name', 'sort_group_id', 'sort_interval_name', 'preproc_params_name', 'team_name', 'sorter', 'sorter_params_name', 'artifact_removed_interval_list_name']
Here is an example of what spikesorting_selection_keys should look like:
spikesorting_selections = [
{
"nwb_file_name": "J1620210531_.nwb",
"sort_group_id": 0,
"sort_interval_name": "raw data valid times no premaze no home",
"preproc_params_name": "franklab_tetrode_hippocampus",
"team_name": "JG_DG",
"sorter": "clusterless_thresholder",
"sorter_params_name": "clusterless_fixed",
"artifact_removed_interval_list_name": "J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times",
},
{
"nwb_file_name": "J1620210531_.nwb",
"sort_group_id": 1,
"sort_interval_name": "raw data valid times no premaze no home",
"preproc_params_name": "franklab_tetrode_hippocampus",
"team_name": "JG_DG",
"sorter": "clusterless_thresholder",
"sorter_params_name": "clusterless_fixed",
"artifact_removed_interval_list_name": "J1620210531_.nwb_1_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times",
},
]
WARNING: Remember that this will only work if you have populated both SpikeSortingSelection and IntervalPositionInfo. You can check this by running the following and making sure there are table entries:
from spyglass.spikesorting import SpikeSortingSelection
SpikeSortingSelection & spikesorting_selections
| nwb_file_name name of the NWB file | sort_group_id identifier for a group of electrodes | sort_interval_name name for this interval | preproc_params_name | team_name | sorter | sorter_params_name | artifact_removed_interval_list_name | import_path optional path to previous curated sorting output |
|---|---|---|---|---|---|---|---|---|
| J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | |
| J1620210531_.nwb | 1 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_1_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times |
Total: 2
Remember to replace the nwb_file_name with your own nwb file name
from spyglass.common import IntervalPositionInfo
IntervalPositionInfo & {
"nwb_file_name": "J1620210531_.nwb",
"position_info_param_name": "default_decoding",
}
| position_info_param_name name for this set of parameters | nwb_file_name name of the NWB file | interval_list_name descriptive name of this interval list | analysis_file_name name of the file | head_position_object_id | head_orientation_object_id | head_velocity_object_id |
|---|---|---|---|---|---|---|
| default_decoding | J1620210531_.nwb | pos 0 valid times | J1620210531_XOKZ4G53LE.nwb | 200c54a9-57dc-4ed4-8e48-85cc9387b922 | 26f4ec71-2f5f-4ef2-82f1-ee3a89048168 | 7f9ffe2f-31e5-47ac-bad9-558440cd5f1d |
| default_decoding | J1620210531_.nwb | pos 1 valid times | J1620210531_B257TS35XJ.nwb | c164cdfd-c328-49b2-b6b4-f8663a17f5d6 | e1f26617-c44b-4464-942d-5e6755606789 | f3c69ac6-5f0a-4622-827b-44a769ce4f77 |
| default_decoding | J1620210531_.nwb | pos 10 valid times | J1620210531_1RICS57YYG.nwb | 2394ec2d-c076-4b11-a40b-91e4b51b81fa | 005080ee-b93f-4b3c-bcd0-595b46fedab2 | 47158c4f-3632-4ce1-9f1e-043f1ea5a5ec |
| default_decoding | J1620210531_.nwb | pos 11 valid times | J1620210531_HZSI1BGTE7.nwb | 295aa187-81cd-4209-b96d-163f12a716ed | 19a95d85-8cb9-4e06-82ee-26bc6df7c8c8 | 9abb39ce-f0e2-4637-b9e9-5f7b6891201d |
| default_decoding | J1620210531_.nwb | pos 12 valid times | J1620210531_M0UMC7TZ49.nwb | ae4c015c-9d5f-4ca3-bbcd-049c1e1ae50b | 1bbc244c-4d85-4e63-9d69-10815e37276c | be8f6ba5-3cc9-4ac3-96af-5c3e84b9851c |
| default_decoding | J1620210531_.nwb | pos 13 valid times | J1620210531_TREA1SOT03.nwb | c4bfe562-f523-495c-99e5-681524661264 | 65023b5e-321b-4d6e-b975-ec03cbd46247 | ce391291-6107-420a-a73d-9a18d343a47a |
| default_decoding | J1620210531_.nwb | pos 14 valid times | J1620210531_93PXV7F4CX.nwb | 8521f813-635d-41cd-8f10-b00f26a7f957 | 75d08124-57f7-481b-ae44-0752a5912c18 | 84ce4054-8c11-49f2-8295-7d8173cb7eb7 |
| default_decoding | J1620210531_.nwb | pos 15 valid times | J1620210531_FUJ1B1OKOA.nwb | e72e5fcd-3d80-463a-a9d1-305e6e99f8db | 6e8dc0e2-ab02-4305-bf2e-e2d208bae77a | b7589719-49ba-48e7-a20b-88a911c3f8a5 |
| default_decoding | J1620210531_.nwb | pos 16 valid times | J1620210531_76HL0SNP59.nwb | b3124712-55db-4b0b-9bdc-77f03a0b8c98 | 08bfd82c-1e7c-4aa7-bd9e-256577b26e9c | 85ffd435-3dc4-49ab-adea-a3327817a51d |
| default_decoding | J1620210531_.nwb | pos 17 valid times | J1620210531_5F8VRT0OC4.nwb | 78b8f7e4-fb90-438d-8551-d15a4d97484a | 7b96d15c-9721-44ea-9ecd-d8a2b60711d6 | 12336768-227e-4c0c-9ade-b3c7ac6c2de3 |
| default_decoding | J1620210531_.nwb | pos 18 valid times | J1620210531_PP7XK1G26S.nwb | 39ab3238-a0dd-4906-95c5-8e534458e371 | ca0f939d-f0a7-4df4-9946-2f605ef5ab76 | 571277d0-885e-4d0e-b802-6ad8047ee3f1 |
| default_decoding | J1620210531_.nwb | pos 19 valid times | J1620210531_AQZHSP2IDM.nwb | b5a21053-3239-4943-aee1-3227b4915d5f | 9f0c61c8-90ed-4a74-873c-e250aa872a2a | 714f6ceb-6d70-474c-8c42-6f9fa3ff77e1 |
...
Total: 21
Now that we've checked those, we can run the function:
populate_mark_indicators(spikesorting_selections)
We can verify that this worked by importing the UnitMarksIndicator table:
from spyglass.decoding import UnitMarksIndicator
UnitMarksIndicator & spikesorting_selections
| curation_id a number correponding to the index of this curation | nwb_file_name name of the NWB file | sort_group_id identifier for a group of electrodes | sort_interval_name name for this interval | preproc_params_name | team_name | sorter | sorter_params_name | artifact_removed_interval_list_name | mark_param_name a name for this set of parameters | interval_list_name descriptive name of this interval list | sampling_rate | analysis_file_name name of the file | marks_indicator_object_id |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 0 valid times | 500.0 | J1620210531_NS19JB65RU.nwb | d0de79cc-f2be-4e86-b4e8-e58a8aef1c02 |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 1 valid times | 500.0 | J1620210531_YP9MXAPE2G.nwb | 7598dc69-a024-4140-8b37-d1342a0e7829 |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 10 valid times | 500.0 | J1620210531_AGOI7DXRQX.nwb | 12a60beb-3ac2-41ae-a091-eebd0b14672c |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 11 valid times | 500.0 | J1620210531_VXGQCG8UYO.nwb | d0d40c96-e5a1-494f-b8a7-bf475304d640 |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 12 valid times | 500.0 | J1620210531_NF8RWZGCSG.nwb | 2b6476a7-143f-4760-bafa-e4959f0e735a |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 13 valid times | 500.0 | J1620210531_F7OHK7UMO3.nwb | 84aea599-dbf1-4d11-bca2-00aa6ae9967a |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 14 valid times | 500.0 | J1620210531_MPXHIH43Y1.nwb | 78224e49-0118-458d-9aee-727db98fc306 |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 15 valid times | 500.0 | J1620210531_YFG1894358.nwb | 721526f8-df6d-442e-84ee-bf7dd6ff8c19 |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 16 valid times | 500.0 | J1620210531_VJ5V0LRJKU.nwb | 39258f97-4508-4068-9b2b-ce419a5d718d |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 17 valid times | 500.0 | J1620210531_NACXLH30KV.nwb | bf312f88-af31-4247-83dc-9633d191401a |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 18 valid times | 500.0 | J1620210531_LFKSWYVVRS.nwb | af89e3e1-71f7-4c7c-a5ef-94d3cc7a1203 |
| 0 | J1620210531_.nwb | 0 | raw data valid times no premaze no home | franklab_tetrode_hippocampus | JG_DG | clusterless_thresholder | clusterless_fixed | J1620210531_.nwb_0_raw data valid times no premaze no home_franklab_tetrode_hippocampus_JG_DG_group_0.8_2000_8_1_artifact_removed_valid_times | default | pos 19 valid times | 500.0 | J1620210531_R9V8JD4IQA.nwb | 3616a7b7-fca2-43bb-9c72-44eb3c22af80 |
...
Total: 42